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A synthetic image viewed from a virtual viewpoint 
above a car is created from images captured by cameras for 
imaging the surroundings of the car. In the synthetic image, 
an illustrated or real image of the car is displayed in the area 
where the car is present. The area which is not imaged by 
any of the cameras is displayed as a blind spot. 
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u = s t ep x*x + o f f X 

v=s t ep y*y + o f f y 

ggo^aibXT^y r (s t e p_x, s t e p_y) <DliS ^ iitc i o 

?>«j t) tB^nsx' y t:> '77^—7' ;V'®^i^i^. l:^^' 7 (step_x 

, s t e p_y ) 0^<btcf&bTMP T l, MPT2, MPT3. MP T 4 
itl^Tzt^. ^^mma id) , (e) , (f) , (g) ® J; ot3^<bt- 

mm (u, v) ;?)iJ^.-7 y trv^^'T^ — r;K7)iBH^^ixfci: ^{*. H5xEb^c¥t7 

^Wicom-^tMi^. m^(o^tjih:^ihti-t $fc. ii5^tbL;^7^y7- (st 

ep_x, s t e p_y ) CI i:<fe-t-^. 



wo 00/64175 PCT/JP0O/D2474 

27 
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-y/H_D ISP) tCF*|^1-i..'^.nb$:*to-5>o ikt^is ,-^na, nb^ (x/ 
W_DISP: l-y/W_DISP) iZf'i'S^ir ^ !^.<D&W.^^tf>tl\t^ CtvA^ 
^sHiR-rS V l^^^^'x — ii'MDOl^^i (u, v) fc-feS. 

S ti 8S H { 3 ¥ tr 55: 11 ® .i: a ^j: M « ^ ;^ © ^ f b i: IE Si 1 3 - Si f S ^ fiK li # ^ 
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m^^m^^ y- )^tv = (txv. t/v. tzv) . n^com^^. nw-mmm?k^u— 

(Vxe. Vye, Vze) ©MJtSf * •7-;i/ (Xw, Yw. Zw) tt. 
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S4 7{i. K^<ll^®li^SUiBJ^^#^E-C-$.^c 
/3v^t. 5!J^ vtD^tlfllJS OtDHlfe (Twist) ^rvi:-r-5^x IHig^fJURvtt, 



cosa, 0 -sin a, 

0 10 
sin a, 0 cosa„ 



10 0 

0 cos /3, - sin )8„ 
0 sin fi^ cos)3. 



cosy^ sin 0 
-siny^ cosy^ 0 
0 0 1 
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v) (D'SLW^zi>^mf^\znii&-r^^^m±<omm (uv, vv) {±, mmcomm^yivm 

Uv = 2 X Sv/Wv - 1 

Vv = 2 X Tv/Hv - 1 ( 4 ) 

W±<}:D. -^fiSlj^cDffiaoiii^ (Sv, tv) {c^T/iS-f ^'7-;i/ H^^i^©3i>: 

(Xw. Yw, Zw) tt. ^ ( 1 ) ~ (4) cfc */^0&mTxv, Tyv, Tzv, 
3tr^5©l5l^av. /3v. yv, M.-^ISiUfv J: t) i*:® (5) S« *?) ^ C i: t- ^ 
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tz-^L. ^ (5) -^mm^G^mm (sv. tv) izna^t^mn^yzeifi^M 
—)\^Ymu^(D3»in.m^ (xw.Yw.zw) jiiT«>¥a®^eiS;ss§fcr o r. 
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25 -TV 



■+K 



vl2 



■>21 



^ + r. 



2T-H, 



vn 



2S„-W, 

3«:^3 'wai ^ ^ SfSZ 



(7) 



J:^Tx St (7) (5) trf^AfSCi >). {g®a)&©-&j 

^©1^^ (Sv, Tv) A^t,, ^l' H^#.I^O¥®©3i):7C^^ (Xw, Yw, 

Zw) ^tiLit>^:Lllifi-^^^. 

;i/ h'^^l^T-© 3»:7c^^ (Xw, Yw, Zw) tr^^*Sf #*><^iy^ 
©&®^©i^^{±5t (1) i:l5l«l^tgll^^t-x #*>«^©ttgx 
Tx. Ty> Tzs a. /?. r ©^'^•^ — ^ Sft A ttS^-T S d i: J: t) ^4{> S CI ii 

0!l;itfx ^ 1 ©ffig^Txl, Tyl, Tzl, [qJ^Sral, rli:m«s 
fifeli^CSiJg (Sv.Tv) trMfS-r?)* ^ ^ 1 ©*^ ^i^^^Xel, Yel, Zel*s,WT 
©^ (8) J: Dlt^-Ct^b 







1^ 






FT" 1 


\ 












T 




2.1 












1 



cosy, -sin)', 0 
siny, cosy, 0 
0 0 1 



10 0 
0 cos /S, sin /3, 
-sin fit cos/3, 



0 



" cosa, 0 sin a, 

0 1 0 
- sin a^ 0 cosa. 



(8) 



S&m^fit\^X. 5t ( 3 ) J; t» 
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Ul = fl/ZelxXel 

VI = fl/ZelxYel ••• ( 9 ) 

»:©iC (10) -c-th^-Ct*. 
SI = Wl/ 2 X (Dv + 1 ) 
Tl = HI/ 2 X (Vv + 1 ) —(10) 

).:i±©#^^{- i t). MSiSjfeiii^oHl^ (Sv. Tv) I3*r«to-r^*^ ^ l ®li 
^©iii^ (Si, Tl) **K)^Ci:As-C-^feo I3!«{r*^ ^ 1 ^ 
ntrMb-Ct) (Sv, Tv) JcMj^>-rs®^^^ (Sn. Tn) Aift^T-^^o II^Jw/^ 

^©ffiP?.. (Sn. Tn) 5t)*^^® * ^ v ffl^OSSHrt C 
;?o-CV>SA»s B05g©il£:*^> < ^tl^ if ©S^irO^fft- J: t? - 

1 o^fcJ±aifeSt;^x *^^#^nii^©]^^ (Sn, Tn) 

i: U-CI24 9 lr^-rct^'&:P3t§*7^''»'^^x.S» c:©^7^;i/{±. ^SS^^J^©lat 

4jiil»?S:(Xc,Zc)i: L> *gR©v^^ ^ (a. c)^ffll,^-C, ^©^7=;i/4ife©^ (1 
-c- © HI ^ S: % X ^ d i: J: t) X ^ g lets?! -e t -i) o 
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^ a- ^ -(11) 

it (1 1) Sfflv^■r^ (5) *'^,vzeS?S5fe-ri»c iiicjio-Cx mmns^(o^^ 

iB^(Di5M(Sv, Tv){c:jt*j5S1-S'7 — 3i>l7C^^ (Xw. Yw, Zw) 

Cffl]ffi^i)»?>. HulBli§®¥S^ T^-'l'iilsl^f-^ ^ti^^m^. 

— ^^^HfS^x^i/ — 

(12) T-^f o 




(12) 
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iSttJtZXt'.t^AsCXc, Yc. ZcK Xia. Ym. Zm:fon(D (a, b, c) ©g$^^o„ 

iii^cD^«t-*r*6-rS'7--»»' Ki^«^«>3i>:7C^# (Xw, Yw, Zw) *H-JSL. 

e,||figlcDiaigSl,Tl%*«6Si:^K:. ^cDli«»^©^*lf IE Lfeiii^^®S:thSt-r S 
^aiEtt. v.vl^>^^5^-.7;w®-&figffii^ (Sv.Tv) t:ti^^m^ (Sn.Tn) ©15^ 

n s u > X i C.V 5^ © s§ ® » 'fe ® "fc ^ M A* ^ ^ < b -c- ^ V ^ T- *> . 
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6. ^^mliim<Dmmmmmmlzis\,^x. 



7 . 
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I-.- 



10. mm(Dmmt:mi^r ^w.m(Dti ^^<Dmmmm^x:f]iii.. :in^(Di3 

fcl±. 1513^ 1 oiami:^7^;u*ss^^, ^ 2 ® iB^i: ^^ty HuBB-^iSK 

11. li^^ 1 0 laiscDii^^s^gjzjbHM:. 
c ^ 1# Sit i: 1- iii^«La ^ g , 



12. ii^^ 1 OlB-®cDiii^5aS^gtc45v^-C. 
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2 2 . m^m2 0 *fctt2 1 tBS^©ffl^^^tta^s^-*^v^■r. 

^ fc * l^m 1 -5 iiiifciifl.31^li o 
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c i: s i^ss[ i: -r a^sas^M o 

2 5. ^*]g2 4le«©Hi^^aa^g^c:i3u^■c, 
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BulB^ 2 © T «v ></a^ — ^' tt. 

-&J5£iiI^©Bl^t3^L. ^^iS^t^WfS-r?. , iBm$ixfcHSiBflif^©®^tc43 

It ^Mii BBJ$ -r -5 * © -e * i) 

2 7. it*]® 2 4iBm©ia^^as^g{r4bH^•c. 

HUlB^ 2 ©V iJ'{4. 

ai^ © Hi m *t liS f S il ^ 7^ - ^ 5: ^ idi f ^ * © T- 5 



2 8. ^i^©^H;&ii]^-r-5^S!t©5!j^ ^©^^iil^^A;t)ii cti?)©* 




wo 00/64175 



PCT/JPOO/02474 



c i: i^gfc i: T ^ Hi^^a^S o 



wo 00/64175 



PCT/JPOO/02474 



1/47 

Fig. 1 



mi 



r 



.1 



mn - 



1 mm 














(b) 



wo 00/64175 



PCT/JPOO/02474 



3/47 

Fig. 3 





wo 00/64175 



PCT/JPOO/02474 




WO 00/64175 



PCT/JPOO/02474 



101 



102 , 

102aOLJ^i^A 



102b^ 



^J2 



> • -7L^ — A t 



1 



7/47 

Fig.7 



210 S*/<^-:^ffi1SSi5 



212 





MPT 



ii!fc<iHi-5 



220 



201 



204 



205 



200 il^&iXSB 



wo 00/64175 



PCT/JPOO/02474 



8/47 



Fig. 8 



S17> 



^ 



NO 



'S11 



'SI 2 



SI 3 



-SI 4 




SI 5 



YES 



wo 00/64175 



PCT/JPOO/02474 



S30 



Tfi 



10/47 



Fig. 10 



S14 mm^m. 



YES 



-S22 
S23 




.S24 



YES 



'S26 



YES 



'S27 



'S28 




'S31 



C ) 



wo 00/64175 



11/47 

Fig. 1 1 



<MP15> 







ymm 




R 


G 


B 


2 


10 


350 


O. 2 


99 


0 


0 


255 


0. 8 



wo 00/64175 



PCT/JPOO/02474 



12/47 



Fig. 12 





Fig. 14 





wo 00/64175 PCT/JPOO/02474 



14/47 



Fig. 1 5 



wo 00/64175 



PCT/JPOO/02474 



Fig. 1 6 



15/47 




V " ^ '™ 



Fig. 17 




wo 00/64175 



PCT/JPOO/02474 



16/47 I 




WO 00/64175 



Fig. 21 



PCT/JPOO/02474 



'i 



(a) 



(b) 



r-5 



wo 00/64175 



PCT/JPOO/02474 



23/47 



Fig. 25 




1 



WO 00/64175 



PCT/JPOO/02474 



Fig. 26 




(0,0) 



-> 



24/47 
(b) 



W MAP 



y H DISP 




(d) 



MAP 




WO 00/64175 



PCT/JPOO/02474 



Fig. 27 



(0,0) 



(a) 

W DISP 



y 

(x.y) 



H DISP 




(off_x,off_y) 



MD 




MAP 



J 



(c) 





wo 00/64175 



PCT/JPOO/02474 



Fig. 28 



26/47 



W DISP 



y 

(X. y) 



\ H DISP 



nZ 







MD 




na d 






n3 
















/ n4 



wo 00/64175 PCT/JPOO/02474 



27/47 

Fig. 29 




"1 



WO 00/64175 



PCT/JPOO/02474 



28/47 

Fig. 30 




wo 00/64175 



PCT/JPOO/02474 



Fig. 31 



29/47 



!ftl,N®a(DaT?>LiiHii 




WO 00/M175 



PCr/JPOO/02474 



33/47 

Fig. 35 



WO 00/64175 PCT/JPOO/02474 



Fig. 36 



L.1 




34/47 



i 



WO 00/64175 



PCT/JPOO/02474 



Fig. 37 35/47 




WO 00/64175 



PCT/JPOO/02474 




WO 00/64175 



PCT/JPOO/02474 



39/47 



Fig. 41 






xmm 






-1 




-1 


0 



5 -s'-V /'^'^«*5*S^x:«^ 



a .A 3 2 22 ?S ?f ?5 ?2 £?? ^ c5 ?5 5§ S 55 S:-^: v.: 







2 


497 


192 


0.3 


3 


202 


236 


0.5 






mm 






99 


360 


44 


1 













316 


215 


1 




WO 00/64175 



PCT/JP00y02474 



Fig. 42 



40/47 



<PT1> 



<PT2> 



<PT3> 



<PT4> 









0 








0 








0 








0 








0 








0 




450 


200 


t 




-1 


-1 


0 







{Km 






-1 


-! 


0 




-1 


-I 


0 




-I 


-1 


0 




150 


280 


05 




490 


280 


OJ 






-1 


0 






-1 


0 





H5H 










-\ 


0 






-\ 


0 






-) 


0 






-1 


0 






-1 


0 




no 


2S0 


f 






-1 


0 



HSU 




HSIH 








-\ 


0 - 




600 


290 








-1 


0 






-1 


0 






-\ 


0 






♦t 


0 






-1 


0 











0 






0 


I 






0 








0 








0 








0 








0 




450 


200 


t 






-» 


0 







-1 


0 






«l 


0 






♦1 


0 






-1 


0 






-1 


0 




4)0 


280 


1 






-1 


0 








0 







mm 






^DH 






mm 






HBH 


mm 


HOI 




HOH 












HDB 






™ 


1^ 


mm 






mm 


mm 




mm 











0 






0 


1 






0 


~\ 






0 


3 






0 


4 






0 


5 






0 


6 






0 


7 






0 











6 


450 


200 


1 







Yff» 




s 


490 


280 


1 



' -1 ■ 



J I 



-I I -I i 0 I 











6 


450 


200 







xsti 


Y^^ 




5 


490 


2t0 


1 







Y3a 1 fi*s 


-1 


-1 


-1 1 0 







Ygff 




99 


360 


44 


1 




PCT/JPOO/02474 



1/47 



0) 



(b) 



PT2l=>!tl5(150» 280) 



PT1|rjl*lS(450, 200) 
PT2ICS*IS(480. 280) pT3lC*rf&(1 10. 250) 



(c) 



(d) 



wo 00/(i4175 



PCT/JPOO/02474 



43/47 



Fig. 45 



<PT5> 



<PT6> 



<PT7> 









MS 


6 


450 


200 


1 











-1 


-1 


-t 


0 











2 


600 


?99 


1 











MS 


6 


450 


200 


1 











-1 


-1 


-I 


0 











-1 


-« 0 











§ 


450 


200 


t 









MS 


-1 


-1 


-1 


0 











99 


360 


44 


1 



wo 00/64175 



PCT/JPOO/02474 



45/47 

Fig. 47 





J 




WO 00/64175 PCT/JPOO/02474 

47/47 

Fig. 49 




F i g. 50 




INTERNATIONAL SEARCH REPORT 



International application No. 

PCT/JPOO/02474 



A. CLASSIHCATION OF SUBJECT MATTER 

Int. CI'' H04N7/18, B60R1/00, G03B15/00, G03B37/00 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 
Int.Cl"' H04N7/18, B60R1/00 



Documentation searched other than minimum documentation to the extent diat such documents arc included in the fields searched 
Jitsuyo Shinan Koho 1922-1996 Toroku Jitsuyo Shinan Kbho 1994-2000 

Kokai Jitsuyo Shinan Koho 1971-2000 Jitsuyo Shinan Toroku Koho 1996-2000 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



X 
Y 
A 



Y 
A 



Y 
A 



Y 
A 



Citation of document, with indication, where appropriate, of the relevant passages 



JP, 58-110334, A (HINO MOTORS, LTD.), 

30 June, 1983 (30.06.83), 

Full text; Figs. 1 to 3 (Family: none) 



JP, 1-123587, A (Mitsubishi Motors Corporation) , 

16 May, 1989 (16.05.89), 

Full text; Figs. 1 to 3 (Family: none) 

JP, 9-180088, A (Nissan Motor Co., Ltd.), 
11 July, 1997 (11.07.97), 

Full text; Figs. 1 to 11 (Family: none) 

JP, 5-238311, A (Mitsubishi Motors Corporation), 

17 September, 1993 (17.09.93), 

Full text;, Figs. 1 to 6 (Family: none) 



JP, 9-305796, A (Canon Inc.), 
28 November, 1997 (28.11.97) 



(Family: none) 



JP, 9-114979, A (Nippon Telegr. & Teleph. Corp. <NTT>) , 
02 May, 1997 (02.05.97) (Family: none) 



Relevant to claim No. 



1,9,17-23,29 
2,5,6,10-16 
3-4,7,8, 
24-28 

2,5, 10-16 
3-4 



2,5,10-16 
3-4 



2,5,6,10-16 
3-4 



1-29 



1-29 



I I Further documents are listed in the continuation of Box C. Q See patent family annex. 



* Special categories of cited documents: 

"A" document deftning the general stale of the ait which is not 

considered U> be of particular relevance 
**£* earlier document but published on or after the international flling 

dale 

"L"* document which may throw doubts on priority claim(s) or which is 
cited to establish the publication date of another citation or other 
special reason (as specified) 

**0'* document referring to an oral disclosure, use, exhibition or other 
means 

"P" document published prior to the international filing date but later 
than the priority date claimed 



**T' later document published after the intemaliona] filing date or 
priority date and not in conflict with the application but cited to 
undcntand the principle or theory underlying the invention 

"X" document of paiticular relevance; the claimed invention cannot be 
considered novel or cannot be considered to involve an Inventive 
step when the document is taken alone 

**y" docinncnt of particular relevance; the claimed invention cannot be 
considered to involve an inventive step when the doomient is 
combined with one or more other such documents, such 
combination being obvious to a pcreon skilled in the ait 
document member of the same patent family 



Date of the actual completion of the international search 
07 July, 2000 (07.07.00) 



Name and mailing address of the ISA/ 

Japanese Patent Office 

Facsiixkile No. 



Date of mailing of the international search report 
25 July, 2000 (25.07.00) 



Authorized officer 



Telephone No. 



Form PCT/ISAy210 (second sheet) (July 1992) 



a 



msiuamif^ pcT/jpoo/02474 



A. m«mn»m (i po ) 

Int.cr H04N7/18, B60R1/00, G03B15/00, G03B37/00 



1IKE«ritofc«/Wa»JEt (SREWfM^S (IPC)) 
Int. CI' HO 4N 7/1 8, B 6 OR 1/0 0 



B^mmmmm^jtwi 1922-1996^ 

B*B^iBllffl»f36i2s« 197 1-2 00 0^ 

0*BS»||ffl»flfe^« 1994-200 0^ 

B:^mnmmMm»&Wi 1996-200 o<p 



HKsa^-c^fflb^s^T^^^^— ^ (x— ^t^^fi^. la^tc^sffl u^bffls) 












X 
Y 

- A 

Y 
A 


JP, 5 8-1 1 0 3 3 4, A (BSSib^X^^^^tt) 3 0. 
1983 (30. 06. 83) ^M, ^1-312 

JP, 1-123587, A i^m^mm.T.m^^'^m i e . 5 

n. 1 98 9 (1 6. 05. 8 9) :^M. i^l -3^1 


1,9. 17-23,29 
2, 5. 6. 10-16 
3-4, 7, 8, 
24-28 

2, 5, 10-16 
3-4 




MR (SftSr^r) ±«oS:ilfci<o, ^m^t t or SWCfc^ffl^i:: 
fpj HRWHBWt?. *»o«jfe«o^!g(oSBSt<c5W« r&j (^-/■?7^:^h7r5:y-:a:ia 


BBISaSSr^TLfcB 

07.07.2000 


aisia2E«^o5ssiB 2 5 07 00 


B*:@!|«fF/r (I SA/ J P) 
Sffi*^ 100-8915 
jK]a»=H^mES«s|§HTB 4« 3 ^ 


SSS^ 03-3581-1 101 


5 P 8 8 3 6 
|7 1 1 


\h» 3 5 8 1 



«33;PCT/I SA/'2 I O (S2-<-v?) (1 9 9 8^7>^) 



jgB^ffig^^ PCTXJ POO/0 24 7 4 



Y I J p. 9-1 80 0 8 8. A H- 7^. 2,5.10-16 
A 11 9 9 7 (1 1.0 7.97) ikH, fRl-1 1® 13-4 

Y UP, 5-2 383 1 1, A (H^ S Sj*xm^^^^) 17. 9 |2.5,6,l(hl6 
A 1^.1 9 9 3 (1 7.09.93) ^K. f^l-60 13-4 

JP. 9-3 0 5 7 9 6, A (^^ / ^-^^^^tfc) 28. 1 1^. 1 1-29 
1 99 7 (2 8. 1 1. 9 7) (7r^y— ^iU 



A J P. 9-11497 9, A (B*m^ 

1 997 (0 2. 0 5. 9 7) (:7T5y— <CU 



tfc) 2. 5M. 1-29 



SSCPCT/I SA/2 1 0 (SB2^— :?(01BIE#) (I9?8¥7fl) 



i 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR STOES 

□ FADED TEXT OR DRAWING 



LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: ] 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




BLURRED OR ILLEGIBLE TEXT OR DRAWING 



□ SKEWED/SLANTED IMAGES 



□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 



□ GRAY SCALE DOCUMENTS 




